Bootsrapping Hypothesis Testings in Credit Risk




Andrija Djurovic

Bootstrapping in Credit Risk

- Generally underutilized in the credit risk area. Particularly useful for testing metrics where there is little consensus on standard error or when statistical testing procedures are absent.

  • Advantages:

    • does not rely on any distributional assumption
    • flexible application across a wide range of metrics
    • useful for identifying bias
    • easy to implement.
  • Disadvantages:

    • computational intensity
    • assumption of independence (although this can be addressed to a certain extent)
    • accuracy concerns for smaller sample sizes.


Example 1: Population Stability Index (1-sided test)

Dataset:

##   Bin Base Target  PSI
## 1   1 0.22   0.35 0.18
## 2   2 0.24   0.31 0.18
## 3   3 0.07   0.05 0.18
## 4   4 0.48   0.29 0.18


Testing Hypothesis:
What is the probability that PSI value is less or equal to 0.15?


Visualization:

## p-value = 21.26%

Example 2: Herfindahl-Hirschman Index (1-sided test)

Dataset:

##         Rating Grade # obs.   DR
## 1   01 (-Inf,0.0199)    202 0.01
## 2 02 [0.0199,0.0263)     54 0.02
## 3 03 [0.0263,0.0369)     96 0.03
## 4 04 [0.0369,0.0903)    204 0.06
## 5   05 [0.0903,0.15)    103 0.11
## 6    06 [0.15,0.197)     41 0.12
## 7     07 [0.197,Inf)     50 0.32


Testing Hypothesis:
What is the probability that HHI value is greater or equal to 0.20?


Visualization:

## p-value = 23.72%

Example 3: Area Under Curve (2-sided test)

Dataset:

## Bootstrapped AUC summary:
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##  0.6769  0.7414  0.7526  0.7525  0.7638  0.8067
## Development sample AUC 79%.
## Application portfolio AUC 75.2%.


Testing Hypothesis:
What is the probability that the application portfolio AUC is equal to 79%?


Figure 1.

Figure 2.

## p-value = 2.26%